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CLAIMS 

What is claimed is: 



1 . A method for generating three-dimensional models from uncalibrated 
5 views comprising acts of: 

- forming a three-dimensional model of at least a portion of a scene 
viewed from an uncalibrated image capturing device location by: 

receiving images from uncalibrated views of the uncalibrated 
image capturing device location; 
10 - extracting features from the images from uncalibrated views of the 

uncalibrated image capturing device location; 
computing correspondence between features from images from the 
uncalibrated views captured from the uncalibrated image capturing 
device location; 

15 • - forming a three-dimensional structure modeling the at least a 

portion of a scene viewed from the uncalibrated image capturing 
device location; 

- iteratively performing the act of forming a three-dimensional model 
viewed from an uncalibrated image capturing device location, for a 

20 subset of the uncalibrated image capturing device locations available; 

- stitching together the three-dimensional models viewed from the 
subset of the uncalibrated image capturing device locations by: 

- finding spatially local persistent feature groupings from the 
uncalibrated views captured at an uncalibrated image capturing 

25 device location; 

- iteratively performing the act of finding spatially local persistent 
feature groupings for a subset of the uncalibrated image capturing 
device locations available; 

- computing correspondence between sets of feature groupings from 
30 two uncalibrated image capturing device locations for a subset of 
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pair-wise combinations of uncalibrated image capturing device 
locations; 

- searching for best matches, whereby multiple three-dimensional 
models from a subset of uncalibrated image capturing device 
locations are thus "stitched" together to form an overall three- 
dimensional model of at least a portion of a scene; and 
- outputting the overall three-dimensional model of at least a portion of 

a scene. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 1, wherein in the act of receiving images from 
uncalibrated views, the images are obtained using at least one uncalibrated 
image capturing device selected from a group consisting of a still camera, 
a video camera, a Magnetic Resonance Imaging (MRI) recording 
mechanism, an ultrasound recording apparatus, an external computer 
connected to the system, an internet connection, an internet web camera, a 
direct satellite link, a video cassette recorder (VCR), a digital versatile 
disc (DVD) player, and imaging recording media used to gather snapshots 
of a desired portion of a scene at multiple uncalibrated views, the imaging 
recording media selected from a group consisting of a Compact Disk 
(CD), a Digital Versatile Disk / Digital Video Disk (DVD), a floppy disk, 
a magnetic tape, a removable hard drive, a printed picture, a scanned 
document, a faxed document, a digital camera, a video cassette, a 
Magnetic Resonance Imaging (MRI) recording media, an ultrasound 
recording media, and a solid-state recording media. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 2, wherein the images from uncalibrated views 
are generated from a group consisting of: images generated by a single 
uncalibrated image capturing device viewing at least a portion of a scene 
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at multiple pan and tilt settings; images captured with multiple 
uncalibrated image capturing devices viewing at least a portion of a scene; 
and images generated by multiple uncalibrated image capturing devices 
viewing at least a portion of a scene at multiple pan and tilt settings. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 3, wherein a portion of a scene comprises at 
least one object, and wherein the images from uncalibrated views are 
formed from a group consisting of: images containing overlapping views 
of a portion of a scene, images containing partially overlapping views of a 
portion of a scene, images containing slightly overlapping views of a 
portion of a scene, and images containing non-overlapping views of a 
portion of a scene. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 4, further comprising a act of identifying and 
eliminating unpaired features prior to computing correspondence between 
features and computing correspondence between sets of feature groupings. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 5, wherein in the act of extracting features from 
the images, the features include at least one of: corner features, high 
entropy points, local edge features, and contour features. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 6, wherein the correspondence between features 
and the correspondence between sets of feature groupings are computed 
by using a technique selected from a group consisting of: probabilistic 
matching, correlation measure, chi-square statistical measure, and dot 
product of feature vectors. 
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8. A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 7, wherein in the act of forming a three- 
dimensional structure from the uncalibrated image capturing device 

5 location, the three-dimensional structure is formed by a "structure from 

motion" algorithm, where motion from the uncalibrated image capturing 
device is computed from the correspondence established from the images 
of uncalibrated views captured at different pan-tilt settings of the 
uncalibrated image capturing device location, and where the "structure 
10 from motion" algorithm simulates a three-dimensional structure modeling 

at least a portion of a scene from the motion of the uncalibrated image 
capturing device. 

9. A method for generating three-dimensional models from uncalibrated 

15 views as set forth in claim 8, wherein the correspondence between features 

and the correspondence between sets of feature groupings are computed 
using a probabilistic matching method, where the probabilistic matching 
method computes probabilities of match between features by using prior 
information representing a portion of a scene, and where the probabilities 

20 of match between features correspond to a posteriori probabilities. 

10. A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 9, wherein in the act of identifying and 
eliminating unpaired features, two unpaired features are identified by 

25 computing and plotting an a posteriori probability relating both features, 

wherein the a posteriori probability relating both features has a flat profile 
when the two features are unpaired. 

11. A method for generating three-dimensional models from uncalibrated 
30 views as set forth in claim 10, wherein the a posteriori probabilities are 
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used to form a correspondence matrix, where a one-to-one correspondence 
between feature groupings from two uncalibrated image capturing device 
locations is established by maximizing the correspondence matrix. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 11, wherein a Sinkhorn normalization process 
is used to form the correspondence matrix. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 12, wherein in the act of searching for the best 
matches, a Ransac robust estimation algorithm is used to find peaks on the 
correspondence matrix, wherein the peaks on the correspondence matrix 
indicate where the three-dimensional models from the uncalibrated image 
capturing device locations are to be stitched together. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 13, wherein the a posteriori probability, 
P(H. | X), is defined by 

K 

P(H i | X) = £ P(Hi I )P(Zx, Mj \ X = X n) 

k=\ 

where 

X= [X lf X N ] and Y= [Y If ... , Y M ] denote two sets of features 
extracted from two images from uncalibrated views; wherein a total of 
iV features were extracted from an uncalibrated view, and a total of M 
features were extracted from another uncalibrated view; 

/i = fl x ,...,{l K represent a set of features extracted from prior 
information, wherein a total of K features were extracted from prior 
information of the portion of the scene; 
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Hi denotes a hypothesis that a feature F/ matches the set of features X, 
wherein i is a variable index with an integer value between 1 and M, 
and wherein i denotes an t h feature within the set of features F; 

%Xfij denotes an event that { X matches Mj }, wherein j is a variable 

5 index with an integer value between 1 and K, and wherein j denotes a 

f h feature within the set of prior features M ; 

P(g x \ X = X n ) denotes a probability of the set of features X 

matching a prior information feature Mj given that the set of features 

X consists of a feature X„, wherein n is a variable index with an 
10 integer value chosen between 1 and N, and wherein n denotes a n th 

feature within the set of features X\ 

P(H i | X, g x ) denotes a probability of a feature Yi matching the 
set of features X given an event occurred (i.e. the set of features 

X matches the prior information feature Mj ); wherein i denotes the i h 

15 feature within the set of features Y and j denotes the f h feature within 

the set of prior features M \ 
and wherein the probability of the set of features X matching the prior 

information feature Mj , P(^x,^ I % = %n) > an d the probability of a 

feature F/ matching the set of features X given an event ^>x^ } occurred, 
20 P(H i | X, £ X9Mj ) , are defined by 

\X = X n )= _ K 1 < X n , Mj > 

and 
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P(Hi I x,t Xt/IJ ) = ^ K < 7„Mj > 



1_ 

where 

E denotes a probabilistic expectation measure, and < . > denotes an 
inner product, where the inner product represents a measure of 
5 similarity. 

15. A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 14, wherein in the act of outputting the overall 
three-dimensional model, the outputting device is selected from a group 

10 consisting of at least one of: a computer monitor, a video camera 

connected to a computer, and a computer readable media used to display 
the overall three-dimensional model of a portion of a scene, the computer 
readable media selected from a group consisting of an imaging Compact 
Disk (CD), a Digital Versatile Disk/ Digital Video Disk (DVD), a floppy 

15 disk, a removable hard drive, a video cassette, and a solid-state recording 

media. 

16. A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 1, wherein the images from uncalibrated views 

20 are generated from a group consisting of: images generated by a single 

uncalibrated image capturing device viewing at least a portion of a scene 
at multiple pan and tilt settings; images captured with multiple 
uncalibrated image capturing devices viewing at least a portion of a scene; 
and images generated by multiple uncalibrated image capturing devices 

25 viewing at least a portion of a scene at multiple pan and tilt settings. 

17. A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 1, wherein a portion of a scene comprises at 
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least one object, and wherein the images from uncalibrated views are 
formed from a group consisting of: images containing overlapping views 
of a portion of a scene, images containing partially overlapping views of a 
portion of a scene, images containing slightly overlapping views of a 
portion of a scene, and images containing non-overlapping views of a 
portion of a scene. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 1, further comprising a act of identifying and 
eliminating unpaired features prior to computing correspondence between 
features and computing correspondence between sets of feature groupings. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 1, wherein in the act of extracting features from 
the images, the features include at least one of: corner features, high 
entropy points, local edge features, and contour features. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 1, wherein in the act of forming a three- 
dimensional structure from the uncalibrated image capturing device 
location, the three-dimensional structure is formed by a "structure from 
motion" algorithm, where motion from the uncalibrated image capturing 
device is computed from the correspondence established from the images 
of uncalibrated views captured at different pan-tilt settings of the 
uncalibrated image capturing device location, and where the "structure 
from motion" algorithm simulates a three-dimensional structure modeling 
at least a portion of a scene from the motion of the uncalibrated image 
capturing device. 
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A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 1, wherein the correspondence between features 
and the correspondence between sets of feature groupings are computed 
by using a technique selected from a group consisting of: probabilistic 
matching, correlation measure, chi-square statistical measure, and dot 
product of feature vectors. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 1, wherein the correspondence between features 
and the correspondence between sets of feature groupings are computed 
using a probabilistic matching method, where the probabilistic matching 
method computers probabilities of match between features by using prior 
information representing a portion of a scene, and where the probabilities 
of match between features correspond to a posteriori probabilities. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 22, wherein in the act of identifying and 
eliminating unpaired features, two unpaired features are identified by 
computing and plotting an a posteriori probability relating both features, 
wherein the a posteriori probability relating both features has a flat profile 
when the two features are unpaired. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 23, wherein the a posteriori probabilities are 
used to form a correspondence matrix, where a one-to-one correspondence 
between feature groupings from two uncalibrated image capturing device 
locations is established by maximizing the correspondence matrix. 
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A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 22, wherein the a posteriori probability, 
P(H S \X),is defined by 

K 

P(Hi I X) = ][>(//,. | x,$ X9lgk )P(Z XjMj \X = X n ) 

k=l 

where 

X = [Xj, Xx] and Y= [Yj, ... , Ym] denote two sets of features 
extracted from two images from uncalibrated views; wherein a total of 
N features were extracted from an uncalibrated view, and a total of M 
features were extracted from another uncalibrated view; 

/J = JJL X ,...,jJ, K represent a set of features extracted from prior 
information, wherein a total of K features were extracted from prior 
information of the portion of the scene; 

/f/ denotes a hypothesis that a feature Y§ matches the set of features X, 
wherein i is a variable index with an integer value between 1 and M, 
and wherein i denotes an t h feature within the set of features F; 

%Xmj denotes an event that { X matches Mj }, wherein j is a variable 

index with an integer value between 1 and K, and wherein j denotes a 
f k feature within the set of prior features jH ; 

P(% Xyf ij I X = X n ) denotes a probability of the set of features AT 

matching a prior information feature Mj given that the set of features 

X consists of a feature X n , wherein n is a variable index with an 
integer value chosen between 1 and N, and wherein n denotes a n th 
feature within the set of features X\ 

P(H i | X 9 £ x ) denotes a probability of a feature Y t matching the 
set of features X given an event %Xn } occurred (i.e. the set of features 
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X matches the prior information feature Mj ); wherein i denotes the f h 

feature within the set of features Y and j denotes the /* feature within 
the set of prior features M ; 
and wherein the probability of the set of features X matching the prior 

information feature Mj ■ P(4x, Mj I x ~ X n ) > and me probability of a 
feature F; matching the set of features X given an event hxpj occurred, 
P{H, | X,^ X /lj ) , are defined by 

and 



Ptf XtMj \ X = X„) = ^ -— < X atf ij > 



10 p ( ff i I X'ix*, ) = -^k—^ < > 

where 

E denotes a probabilistic expectation measure, and < . > denotes an 
inner product, where the inner product represents a measure of 
similarity. 

15 

26. A method for generating three-dimensional models from uncalibrated 

views as set forth in claim 1 , wherein a Sinkhorn normalization process is 
used to form the correspondence matrix. 

20 27. A method for generating three-dimensional models from uncalibrated 

views as set forth in claim 1, wherein in the act of searching for the best 
matches, a Ransac robust estimation algorithm is used to find peaks on the 
correspondence matrix, wherein the peaks on the correspondence matrix 
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indicate where the three-dimensional models from the uncalibrated image 
capturing device locations are to be stitched together. 

A method for generating three-dimensional models from uncalibrated 
views as set forth in claim 1 , wherein in the act of outputting the overall 
three-dimensional model, the outputting device is selected from a group 
consisting of at least one of: a computer monitor, a video camera 
connected to a computer, and a computer readable media used to display 
the overall three-dimensional model of a portion of a scene, the computer 
readable media selected from a group consisting of an imaging Compact 
Disk (CD), a Digital Versatile Disk/ Digital Video Disk (DVD), a floppy 
disk, a removable hard drive, a video cassette, and a solid-state recording 
media. 

A system for generating three-dimensional models from uncalibrated 
views, the system comprising: 

a computer system including a processor, a memory coupled with the 
processor, an input coupled with the processor for receiving images from 
uncalibrated views of an uncalibrated image capturing device location, the 
computer system further comprising means, residing in its processor and 
memory for: 

- forming a three-dimensional model of at least a portion of a scene 
viewed from an uncalibrated image capturing device location by: 

extracting features from the images from uncalibrated views of the 

uncalibrated image capturing device location; 
- computing correspondence between features from images from the 

uncalibrated views captured from the uncalibrated image capturing 

device location; 
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- forming a three-dimensional structure modeling the at least a 
portion of a scene viewed from the uncalibrated image capturing 
device location; 

- iteratively performing the means for forming a three-dimensional 
model viewed from an uncalibrated image capturing device location, 
for a subset of the uncalibrated image capturing device locations 
available; 

- stitching together the three-dimensional models viewed from the 
subset of the uncalibrated image capturing device locations by: 

- finding spatially local persistent feature groupings from the 
uncalibrated views captured at an uncalibrated image capturing 
device location; 

- iteratively performing the means for finding spatially local 
persistent feature groupings for a subset of the uncalibrated image 
capturing device locations available; 

- computing correspondence between sets of feature groupings from 
two uncalibrated image capturing device locations for a subset of 
pair-wise combinations of uncalibrated image capturing device 
locations; 

- searching for best matches, whereby multiple three-dimensional 
models from a subset of uncalibrated image capturing device 
locations are thus "stitched" together to form an overall three- 
dimensional model of at least a portion of a scene; and 

- outputting the overall three-dimensional model of at least a portion of 
a scene. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 29, wherein in the means for receiving images 
from uncalibrated views, the images are obtained using at least one 
uncalibrated image capturing device selected from a group consisting of a 
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still camera, a video camera, a Magnetic Resonance Imaging (MRI) 
recording mechanism, an ultrasound recording apparatus, an external 
computer connected to the system, an internet connection, an internet web 
camera, a direct satellite link, a video cassette recorder (VCR), a digital 
versatile disc (DVD) player, and imaging recording media used to gather 
snapshots of a desired portion of a scene at multiple uncalibrated views, 
the imaging recording media selected from a group consisting of a 
Compact Disk (CD), a Digital Versatile Disk / Digital Video Disk (DVD), 
a floppy disk, a magnetic tape, a removable hard drive, a printed picture, a 
scanned document, a faxed document, a digital camera, a video cassette, a 
Magnetic Resonance Imaging (MRI) recording media, an ultrasound 
recording media, and a solid-state recording media. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 30, wherein the images from uncalibrated views 
are generated from a group consisting of: images generated by a single 
uncalibrated image capturing device viewing at least a portion of a scene 
at multiple pan and tilt settings; images captured with multiple 
uncalibrated image capturing devices viewing at least a portion of a scene; 
and images generated by multiple uncalibrated image capturing devices 
viewing at least a portion of a scene at multiple pan and tilt settings. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 31, wherein a portion of a scene comprises at 
least one object, and wherein the images from uncalibrated views are 
formed from a group consisting of: images containing overlapping views 
of a portion of a scene, images containing partially overlapping views of a 
portion of a scene, images containing slightly overlapping views of a 
portion of a scene, and images containing non-overlapping views of a 
portion of a scene. 
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A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 32, further comprising a means for identifying 
and eliminating unpaired features prior to computing correspondence 
between features and computing correspondence between sets of feature 
groupings. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 33, wherein in the means for extracting features 
from the images, the features include at least one of: corner features, high 
entropy points, local edge features, and contour features. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 34, wherein the correspondence between 
features and the correspondence between sets of feature groupings are 
computed by using a technique selected from a group consisting of: 
probabilistic matching, correlation measure, chi-square statistical measure, 
and dot product of feature vectors. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 35, wherein in the means for forming a three- 
dimensional structure from the uncalibrated image capturing device 
location, the three-dimensional structure is formed by a "structure from 
motion" algorithm, where motion from the uncalibrated image capturing 
device is computed from the correspondence established from the images 
of uncalibrated views captured at different pan-tilt settings of the 
uncalibrated image capturing device location, and where the "structure 
from motion" algorithm simulates a three-dimensional structure modeling 
at least a portion of a scene from the motion of the uncalibrated image 
capturing device. 
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A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 36, wherein the correspondence between 
features and the correspondence between sets of feature groupings are 
computed using a probabilistic matching method, where the probabilistic 
matching method computes probabilities of match between features by 
using prior information representing a portion of a scene, and where the 
probabilities of match between features correspond to a posteriori 
probabilities. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 37, wherein in the means for identifying and 
eliminating unpaired features, two unpaired features are identified by 
computing and plotting an a posteriori probability relating both features, 
wherein the a posteriori probability relating both features has a flat profile 
when the two features are unpaired. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 38, wherein the a posteriori probabilities are 
used to form a correspondence matrix, where a one-to-one correspondence 
between feature groupings from two uncalibrated image capturing device 
locations is established by maximizing the correspondence matrix. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 39, wherein a Sinkhorn normalization process 
is used to form the correspondence matrix. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 40, wherein in the means for searching for the 
best matches, a Ransac robust estimation algorithm is used to find peaks 



Page 69 of 90 



METHOD AND APPARATUS FOR 
GENERATING THREE- 
DIMENSIONAL MODELS FROM 
UNCALD3RATED VIEWS 

on the correspondence matrix, wherein the peaks on the correspondence 
matrix indicate where the three-dimensional models from the uncalibrated 
image capturing device locations are to be stitched together. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 41, wherein the a posteriori probability, 
P(H, | X), is defined by 

P(H, | X) = | x,z XtMt )P(4 X , M . | X = X„) 

k=\ 

where 

X = [X], X N ] and F= [Y h ... , Y M ] denote two sets of features 
extracted from two images from uncalibrated views; wherein a total of 
N features were extracted from an uncalibrated view, and a total of M 
features were extracted from another uncalibrated view; 

fJL = fl x represent a set of features extracted from prior 

information, wherein a total of K features were extracted from prior 
information of the portion of the scene; 

Hi denotes a hypothesis that a feature Y$ matches the set of features X, 
wherein i is a variable index with an integer value between 1 and AT, 
and wherein i denotes an ( h feature within the set of features F; 

%Xnj denotes an event that { X matches Mj }, wherein j is a variable 

index with an integer value between 1 and K, and wherein j denotes a 
f h feature within the set of prior features M ; 

P(^ XtMj I % = X n ) denotes a probability of the set of features X 

matching a prior information feature Mj given that the set of features 
X consists of a feature X n , wherein n is a variable index with an 
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integer value chosen between 1 and N 9 and wherein n denotes a n th 
feature within the set of features X\ 

P(H i | X 9 ij x ) denotes a probability of a feature Yi matching the 
set of features X given an event 4xmj occurred (i.e. the set of features 

X matches the prior information feature Mj ); wherein i denotes the ( h 

feature within the set of features Y and j denotes the /* feature within 
the set of prior features M ; 
and wherein the probability of the set of features A' matching the prior 

information feature Mj , PiZx^ I X = X n ) , and the probability of a 

feature Yi matching the set of features X given an event E>xn } occurred, 
P{H t | X, % Xfij ) , are defined by 

J_ 

and 



Ptf XtMj \X = X„) = -^ <X a ,pj> 



PiH, | X£ x ) = — j < Y^fij > 

where 

E denotes a probabilistic expectation measure, and < . > denotes an 
inner product, where the inner product represents a measure of 
similarity. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 42, wherein in the means for outputting the 
overall three-dimensional model, the outputting device is selected from a 
group consisting of at least one of: a computer monitor, a video camera 
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connected to a computer, and a computer readable media used to display 
the overall three-dimensional model of a portion of a scene, the computer 
readable media selected from a group consisting of an imaging Compact 
Disk (CD), a Digital Versatile Disk/ Digital Video Disk (DVD), a floppy 
disk, a removable hard drive, a video cassette, and a solid-state recording 
media. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 29, wherein the images from uncalibrated views 
are generated from a group consisting of: images generated by a single 
uncalibrated image capturing device viewing at least a portion of a scene 
at multiple pan and tilt settings; images captured with multiple 
uncalibrated image capturing devices viewing at least a portion of a scene; 
and images generated by multiple uncalibrated image capturing devices 
viewing at least a portion of a scene at multiple pan and tilt settings. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 29, wherein a portion of a scene comprises at 
least one object, and wherein the images from uncalibrated views are 
formed from a group consisting of: images containing overlapping views 
of a portion of a scene, images containing partially overlapping views of a 
portion of a scene, images containing slightly overlapping views of a 
portion of a scene, and images containing non-overlapping views of a 
portion of a scene. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 29, further comprising a means for identifying 
and eliminating unpaired features prior to computing correspondence 
between features and computing correspondence between sets of feature 
groupings. 
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47. A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 29, wherein in the means for extracting features 
from the images, the features include at least one of: corner features, high 

5 entropy points, local edge features, and contour features. 

48. A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 29, wherein in the means for forming a three- 
dimensional structure from the uncalibrated image capturing device 

10 location, the three-dimensional structure is formed by a "structure from 

motion" algorithm, where motion from the uncalibrated image capturing 
device is computed from the correspondence established from the images 
of uncalibrated views captured at different pan-tilt settings of the 
uncalibrated image capturing device location, and where the "structure 

15 from motion" algorithm simulates a three-dimensional structure modeling 

at least a portion of a scene from the motion of the uncalibrated image 
capturing device. 

49. A system for generating three-dimensional models from uncalibrated 
20 views as set forth in claim 29, wherein the correspondence between 

features and the correspondence between sets of feature groupings are 
computed by using a technique selected from a group consisting of: 
probabilistic matching, correlation measure, chi-square statistical measure, 
and dot product of feature vectors. 

25 

50. A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 29, wherein the correspondence between 
features and the correspondence between sets of feature groupings are 
computed using a probabilistic matching method, where the probabilistic 

30 matching method computes probabilities of match between features by 
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using prior information representing a portion of a scene, and where the 
probabilities of match between features correspond to a posteriori 
probabilities. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 50, wherein in the means for identifying and 
eliminating unpaired features, two unpaired features are identified by 
computing and plotting an a posteriori probability relating both features, 
wherein the a posteriori probability relating both features has a flat profile 
when the two features are unpaired. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 51, wherein the a posteriori probabilities are 
used to form a correspondence matrix, where a one-to-one correspondence 
between feature groupings from two uncalibrated image capturing device 
locations is established by maximizing the correspondence matrix. 

A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 50, wherein the a posteriori probability, 
P(H i \X) 9 is defined by 

PWi I X) = | x^ Mk )Ptf x , Mj \X = X n ) 

k=\ 

where 

X= [X If Xm] and Y= [Fy, ... , Ym] denote two sets of features 
extracted from two images from uncalibrated views; wherein a total of 
iV features were extracted from an uncalibrated view, and a total of M 
features were extracted from another uncalibrated view; 
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fj> = H x 9 ... 9 Jl K represent a set of features extracted from prior 
information, wherein a total of K features were extracted from prior 
information of the portion of the scene; 

Hi denotes a hypothesis that a feature Yi matches the set of features X, 
5 wherein i is a variable index with an integer value between 1 and M 9 

and wherein i denotes an t h feature within the set of features Y\ 

%X/*j denotes an event that { X matches Mj }, wherein j is a variable 

index with an integer value between 1 and K, and wherein j denotes a 
f h feature within the set of prior features M \ 

10 P(€x y tij I % = X n ) denotes a probability of the set of features X 

matching a prior information feature Mj given that the set of features 

X consists of a feature X ny wherein n is a variable index with an 
integer value chosen between 1 and N, and wherein n denotes a n th 
feature within the set of features X\ 

15 P(fli I X> %x,mj ) denotes a probability of a feature Y, matching the 

set of features X given an event £>xn } occurred (i.e. the set of features 

X matches the prior information feature Mj ); wherein i denotes the ( h 

feature within the set of features Fand j denotes the f l feature within 
the set of prior features M \ 
20 and wherein the probability of the set of features X matching the prior 

information feature Mj , P(E> X ^ } \ X = X n ) , and the probability of a 
feature Yi matching the set of features X given an event ^x/jj occurred, 
P(H i | XiZxjij ) > are defined by 
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nix*, \X'X m )m——l <X m9 fij> 



and 



P(H, | X,4 X ) = ! <Y„fj> 

where 

5 E denotes a probabilistic expectation measure, and < . > denotes an 

inner product, where the inner product represents a measure of 
similarity. 

54. A system for generating three-dimensional models from uncalibrated 

10 views as set forth in claim 29, wherein a Sinkhorn normalization process 

is used to form the correspondence matrix. 

55. A system for generating three-dimensional models from uncalibrated 
views as set forth in claim 29, wherein in the means for searching for the 

1 5 best matches, a Ransac robust estimation algorithm is used to find peaks 

on the correspondence matrix, wherein the peaks on the correspondence 
matrix indicate where the three-dimensional models from the uncalibrated 
image capturing device locations are to be stitched together. 

20 56. A system for generating three-dimensional models from uncalibrated 

views as set forth in claim 29, wherein in the means for outputting the 
overall three-dimensional model, the outputting device is selected from a 
group consisting of at least one of: a computer monitor, a video camera 
connected to a computer, and a computer readable media used to display 

25 the overall three-dimensional model of a portion of a scene, the computer 

readable media selected from a group consisting of an imaging Compact 
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Disk (CD), a Digital Versatile Disk/ Digital Video Disk (DVD), a floppy 
disk, a removable hard drive, a video cassette, and a solid-state recording 
media. 

A computer program product for generating three-dimensional models 
from uncalibrated views, the computer program product comprising 
means, stored on a computer readable medium for: 

- forming a three-dimensional model of at least a portion of a scene 
viewed from an uncalibrated image capturing device location by: 

- receiving images from uncalibrated views of the uncalibrated 
image capturing device location; 

- extracting features from the images from uncalibrated views of the 
uncalibrated image capturing device location; 

- computing correspondence between features from images from the 
uncalibrated views captured from the uncalibrated image capturing 
device location; 

- forming a three-dimensional structure modeling the at least a 
portion of a scene viewed from the uncalibrated image capturing 
device location; 

- iteratively performing the means for forming a three-dimensional 
model viewed from an uncalibrated image capturing device location, 
for a subset of the uncalibrated image capturing device locations 
available; 

- stitching together the three-dimensional models viewed from the 
subset of the uncalibrated image capturing device locations by: 

- finding spatially local persistent feature groupings from the 
uncalibrated views captured at an uncalibrated image capturing 
device location; 
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- iteratively performing the means for finding spatially local 
persistent feature groupings for a subset of the uncalibrated image 
capturing device locations available; 

- computing correspondence between sets of feature groupings from 
two uncalibrated image capturing device locations for a subset of 
pair-wise combinations of uncalibrated image capturing device 
locations; 

- searching for best matches, whereby multiple three-dimensional 
models from a subset of uncalibrated image capturing device 
locations are thus "stitched" together to form an overall three- 
dimensional model of at least a portion of a scene; and 

- outputting the overall three-dimensional model of at least a portion of 
a scene. 

A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 57, wherein in the means for 
receiving images from uncalibrated views, the images are obtained using 
at least one uncalibrated image capturing device selected from a group 
consisting of a still camera, a video camera, a Magnetic Resonance 
Imaging (MRI) recording mechanism, an ultrasound recording apparatus, 
an external computer connected to the system, an internet connection, an 
internet web camera, a direct satellite link, a video cassette recorder 
(VCR), a digital versatile disc (DVD) player, and imaging recording 
media used to gather snapshots of a desired portion of a scene at multiple 
uncalibrated views, the imaging recording media selected from a group 
consisting of a Compact Disk (CD), a Digital Versatile Disk / Digital 
Video Disk (DVD), a floppy disk, a magnetic tape, a removable hard 
drive, a printed picture, a scanned document, a faxed document, a digital 
camera, a video cassette, a Magnetic Resonance Imaging (MRI) recording 
media, an ultrasound recording media, and a solid-state recording media. 
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59. A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 58, wherein the images from 
uncalibrated views are generated from a group consisting of: images 

5 generated by a single uncalibrated image capturing device viewing at least 

a portion of a scene at multiple pan and tilt settings; images captured with 
multiple uncalibrated image capturing devices viewing at least a portion of 
a scene; and images generated by multiple uncalibrated image capturing 
devices viewing at least a portion of a scene at multiple pan and tilt 
10 settings. 

60. A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 59, wherein a portion of a 
scene comprises at least one object, and wherein the images from 

15 uncalibrated views are formed from a group consisting of: images 

containing overlapping views of a portion of a scene, images containing 
partially overlapping views of a portion of a scene, images containing 
slightly overlapping views of a portion of a scene, and images containing 
non-overlapping views of a portion of a scene. 

20 

61. A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 60, further comprising a 
means for identifying and eliminating unpaired features prior to 
computing correspondence between features and computing 

25 correspondence between sets of feature groupings. 

62. A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 61, wherein in the means for 
extracting features from the images, the features include at least one of: 
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corner features, high entropy points, local edge features, and contour 
features. 



63. A computer program product for generating three-dimensional models 
5 from uncalibrated views as set forth in claim 62, wherein the 

correspondence between features and the correspondence between sets of 
feature groupings are computed by using a technique selected from a 
group consisting of: probabilistic matching, correlation measure, chi- 
square statistical measure, and dot product of feature vectors. 

10 

64. A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 63, wherein in the means for 
forming a three-dimensional structure from the uncalibrated image 
capturing device location, the three-dimensional structure is formed by a 

15 "structure from motion" algorithm, where motion from the uncalibrated 

image capturing device is computed from the correspondence established 
from the images of uncalibrated views captured at different pan-tilt 
settings of the uncalibrated image capturing device location, and where the 
"structure from motion" algorithm simulates a three-dimensional structure 

20 modeling at least a portion of a scene from the motion of the uncalibrated 

image capturing device. 

65. A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 64, wherein the 

25 correspondence between features and the correspondence between sets of 

feature groupings are computed using a probabilistic matching method, 
where the probabilistic matching method computes probabilities of match 
between features by using prior information representing a portion of a 
scene, and where the probabilities of match between features correspond 

30 to a posteriori probabilities. 
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A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 65, wherein in the means for 
identifying and eliminating unpaired features, two unpaired features are 
identified by computing and plotting an a posteriori probability relating 
both features, wherein the a posteriori probability relating both features 
has a flat profile when the two features are unpaired. 

A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 66, wherein the a posteriori 
probabilities are used to form a correspondence matrix, where a one-to- 
one correspondence between feature groupings from two uncalibrated 
image capturing device locations is established by maximizing the 
correspondence matrix. 

A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 67, wherein a Sinkhorn 
normalization process is used to form the correspondence matrix. 

A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 68, wherein in the means for 
searching for the best matches, a Ransac robust estimation algorithm is 
used to find peaks on the correspondence matrix, wherein the peaks on the 
correspondence matrix indicate where the three-dimensional models from 
the uncalibrated image capturing device locations are to be stitched 
together. 
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70. A computer program product for generating three-dimensional models 

from uncalibrated views as set forth in claim 69, wherein the a posteriori 
probability, P(H, \X),is defined by 

P(H i | X) = 1 X,t XiMi )P^x, tl) \X-X m ) 

5 where 

X= [Xj, X N ] and Y= [Y It ... , Y M ] denote two sets of features 
extracted from two images from uncalibrated views; wherein a total of 
N features were extracted from an uncalibrated view, and a total of M 
features were extracted from another uncalibrated view; 

10 M = Mi )"*)Mk represent a set of features extracted from prior 

information, wherein a total of K features were extracted from prior 
information of the portion of the scene; 

Hi denotes a hypothesis that a feature Yi matches the set of features X, 
wherein i is a variable index with an integer value between 1 and M 9 
1 5 and wherein i denotes an t h feature within the set of features Y\ 

%Xmj denotes an event that { X matches Mj }, wherein j is a variable 

index with an integer value between 1 and K, and wherein j denotes a 
f h feature within the set of prior features M \ 
P(4x,mj I X - X n ) denotes a probability of the set of features X 

20 matching a prior information feature Mj given that the set of features 

X consists of a feature X„, wherein n is a variable index with an 
integer value chosen between 1 and TV, and wherein n denotes a n th 
feature within the set of features X\ 

P(H i | X 9 g x ) denotes a probability of a feature Yi matching the 
25 set of features X given an event ^x^j occurred (i.e. the set of features 
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X matches the prior information feature Mj ); wherein i denotes the C* 

feature within the set of features Fand j denotes the / feature within 
the set of prior features JU ; 
and wherein the probability of the set of features X matching the prior 

information feature Mj , P(£x, Mj I ^ ~ ^„ ) , and the probability of a 
feature F,- matching the set of features X given an event ^ occurred, 
P(H t | X,£ X mj ) , are defined by 

J_ 

and 



Ptfx,^ \x = x„) = ^ — < X.,Mj > 



10 p << H i I X'ix t/IJ ) = ^r- 2 < *i>fj > 

J dk __ l E<Y i ,H k > 

where 

E denotes a probabilistic expectation measure, and < . > denotes an 
inner product, where the inner product represents a measure of 
similarity. 

15 

71. A computer program product for generating three-dimensional models 

from uncalibrated views as set forth in claim 70, wherein in the means for 
receiving images from uncalibrated views, the images are obtained using 
at least one uncalibrated image capturing device selected from a group 
20 consisting of a still camera, a video camera, a Magnetic Resonance 

Imaging (MRI) recording mechanism, an ultrasound recording apparatus, 
an external computer connected to the system, an internet connection, an 
internet web camera, a direct satellite link, a video cassette recorder 
(VCR), a digital versatile disc (DVD) player, and imaging recording 
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media used to gather snapshots of a desired portion of a scene at multiple 
uncalibrated views, the imaging recording media selected from a group 
consisting of a Compact Disk (CD), a Digital Versatile Disk / Digital 
Video Disk (DVD), a floppy disk, a magnetic tape, a removable hard 
drive, a printed picture, a scanned document, a faxed document, a digital 
camera, a video cassette, a Magnetic Resonance Imaging (MRI) recording 
media, an ultrasound recording media, and a solid-state recording media. 

A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 57, wherein the images from 
uncalibrated views are generated from a group consisting of: images 
generated by a single uncalibrated image capturing device viewing at least 
a portion of a scene at multiple pan and tilt settings; images captured with 
multiple uncalibrated image capturing devices viewing at least a portion of 
a scene; and images generated by multiple uncalibrated image capturing 
devices viewing at least a portion of a scene at multiple pan and tilt 
settings. 

A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 57, wherein a portion of a 
scene comprises at least one object, and wherein the images from 
uncalibrated views are formed from a group consisting of: images 
containing overlapping views of a portion of a scene, images containing 
partially overlapping views of a portion of a scene, images containing 
slightly overlapping views of a portion of a scene, and images containing 
non-overlapping views of a portion of a scene. 

A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 57, further comprising a 
means for identifying and eliminating unpaired features prior to 
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computing correspondence between features and computing 
correspondence between sets of feature groupings. 

A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 57, wherein in the means for 
extracting features from the images, the features include at least one of: 
corner features, high entropy points, local edge features, and contour 
features. 

A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 57, wherein in the means for 
forming a three-dimensional structure from the uncalibrated image 
capturing device location, the three-dimensional structure is formed by a 
"structure from motion" algorithm, where motion from the uncalibrated 
image capturing device is computed from the correspondence established 
from the images of uncalibrated views captured at different pan-tilt 
settings of the uncalibrated image capturing device location, and where the 
"structure from motion" algorithm simulates a three-dimensional structure 
modeling at least a portion of a scene from the motion of the uncalibrated 
image capturing device. 

A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 57, wherein the 
correspondence between features and the correspondence between sets of 
feature groupings are computed by using a technique selected from a 
group consisting of: probabilistic matching, correlation measure, chi- 
square statistical measure, and dot product of feature vectors. 

A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 57, wherein the 
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correspondence between features and the correspondence between sets of 
feature groupings are computed using a probabilistic matching method, 
where the probabilistic matching method computes probabilities of match 
between features by using prior information representing a portion of a 
scene, and where the probabilities of match between features correspond 
to a posteriori probabilities. 

A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 78, wherein in the means for 
identifying and eliminating unpaired features, two unpaired features are 
identified by computing and plotting an a posteriori probability relating 
both features, wherein the a posteriori probability relating both features 
has a flat profile when the two features are unpaired. 

A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 79, wherein the a posteriori 
probabilities are used to form a correspondence matrix, where a one-to- 
one correspondence between feature groupings from two uncalibrated 
image capturing device locations is established by maximizing the 
correspondence matrix. 

A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 78, wherein the a posteriori 
probability, P(H i | X) , is defined by 

P{H t | X) = | X£ x%lik )P{$ X ^ \ x = x n) 

where 

X= [X If X N ] and Y= [Y Ir ... , Y M ] denote two sets of features 
extracted from two images from uncalibrated views; wherein a total of 
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N features were extracted from an uncalibrated view, and a total of M 
features were extracted from another uncalibrated view; 

fi — fi x ,...)JJ K represent a set of features extracted from prior 
information, wherein a total of K features were extracted from prior 
5 information of the portion of the scene; 

///denotes a hypothesis that a feature Y { matches the set of features X, 
wherein i is a variable index with an integer value between / and M, 
and wherein i denotes an t h feature within the set of features Y; 

%Xnj denotes an event that { X matches Mj }> wherein j is a variable 

10 index with an integer value between 1 and K, and wherein j denotes a 

f h feature within the set of prior features M ; 

P(£x if ij I % = X n ) denotes a probability of the set of features X 

matching a prior information feature Mj given that the set of features 

AT consists of a feature X„, wherein n is a variable index with an 
15 integer value chosen between 1 and N, and wherein n denotes a n th 

feature within the set of features X; 

P(H i | X,^ X Mj ) denotes a probability of a feature Y§ matching the 
set of features X given an event ^Xpi } occurred (i.e. the set of features 

X matches the prior information feature Mj ); wherein i denotes the i h 

20 feature within the set of features Fand j denotes the f h feature within 

the set of prior features M ; 
and wherein the probability of the set of features X matching the prior 

information feature Mj , P(4x, Mj I ^ ~ ^ n ) , and the probability of a 
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feature Y t matching the set of features X given an event E>xn } occurred, 
P(H i | X, 4 X Mj ) , are defined by 

and 

J_ 

where 

is denotes a probabilistic expectation measure, and < . > denotes an 
inner product, where the inner product represents a measure of 
similarity. 



PV*i I x,4 x ^ ) = ^ K < Y ^J > 



82. A computer program product for generating three-dimensional models 
from uncalibrated views as set forth in claim 57, wherein a Sinkhorn 
normalization process is used to form the correspondence matrix. 

15 83. A computer program product for generating three-dimensional models 

from uncalibrated views as set forth in claim 57, wherein in the means for 
searching for the best matches, a Rarisac robust estimation algorithm is 
used to find peaks on the correspondence matrix, wherein the peaks on the 
correspondence matrix indicate where the three-dimensional models from 

20 the uncalibrated image capturing device locations are to be stitched 

together. 

84. A computer program product for generating three-dimensional models 

from uncalibrated views as set forth in claim 57, wherein in the means for 
25 outputting the overall three-dimensional model, the outputting device is 
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selected from a group consisting of at least one of: a computer monitor, a 
video camera connected to a computer, and a computer readable media 
used to display the overall three-dimensional model of a portion of a 
scene, the computer readable media selected from a group consisting of an 
imaging Compact Disk (CD), a Digital Versatile Disk/ Digital Video Disk 
(DVD), a floppy disk, a removable hard drive, a video cassette, and a 
solid-state recording media. 
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